Ensembles as a Sequence of Classifiers

Lars Asker Richard Maclin Jet Propulsion Laboratory Department of Computer Science M/S 525-3660 University of Minnesota Pasadena, California 91109-8099 Duluth, Minnesota 55812-2496 Abstract An ensemble is a classi er created by combining the predictions of multiple component classi ers. We present a new method for combining classi ers into an ensemble based on a simple estimation of each classi er's competence. The classi ers are grouped into an ordered list where each classi er has a corresponding threshold. To classify an example, the rst classi er on the list is consulted and if that classi er's con dence for predicting the example is above the classi er's threshold, then that classi er's prediction is used. Otherwise, the next classi er and its threshold is consulted and so on. If none of the classi ers predicts the example above its con dence threshold then the class of the example is predicted by averaging all of the component classi er predictions. The key to this method is the selection of the condence threshold for each classi er. We have implemented this method in a system called Sequel which has been applied to the task of recognizing volcanos in SAR images of Venus. In this domain, Sequel outperforms each individual classi er as well as the simple approach of using an ensemble constructed from the average prediction of all the classi ers.